Running head: Preschool mathematics curriculum Effects of a Preschool Mathematics Curriculum: Summary Research on the Building Blocks Project

This study evaluated the efficacy of a preschool mathematics program based on a comprehensive model of developing research-based software and print curricula. Building Blocks, funded by the National Science Foundation, is a curriculum development project focused on creating researchbased, technology-enhanced mathematics materials for PreK through grade 2. In this article, we describe the underlying principles, development, and initial summary evaluation of the first set of resulting materials, as they were implemented in classrooms teaching children at risk for later school failure. Experimental and comparison classrooms included two principal types of public preschool programs serving low-income families, state funded and Head Start pre-kindergarten programs. Children in all classrooms were preand post-tested with an individual assessment based on the curriculum's hypothesized learning trajectories. The experimental treatment group score increased significantly more than the comparison group score. Effect sizes comparing posttest scores of the experiment group to those of the comparison group were .85 for number and 1.44 for geometry, and effect sizes comparing the experimental group’s pretest and posttest scores were 1.71 for number and 2.12 for geometry. Thus, achievement gains of the experimental group were comparable to the sought-after 2-sigma effect of individual tutoring. This study contributes to research showing that focused early mathematical interventions help young children develop a foundation of informal mathematics knowledge, especially for children at risk for later school failure. Preschool mathematics curriculum 2 Effects of a Preschool Mathematics Curriculum: Summary Research on the Building Blocks Project Government agencies have recently emphasized the importance of evidence-based instructional materials (e.g., Feuer, Towne, & Shavelson, 2002; Reeves, 2002). However, the ubiquity and multifariousness of publishers’ claims of research-based curricula, in conjunction with the ambiguous nature of the phrase “research-based,” discourages scientific approaches to curriculum development (and allows the continued dominance of non-scientific “market research”) and undermines attempts to create a shared research foundation for the creation of classroom curricula (Battista & Clements, 2000; Clements, 2002; Clements & Battista, 2000). Once produced, curricula are rarely evaluated scientifically. Less than 2% of research studies concerned the effects of textbooks (Senk & Thompson, 2003), even though these books predominate mathematics curriculum materials in U.S. classrooms and to a great extent determine teaching practices (Goodlad, 1984), even in the context of reform efforts (Grant, Peterson, & Shojgreen-Downer, 1996). This study is one of several coordinated efforts to assess the efficacy of a curriculum that was designed and evaluated according to specific criteria for both the development and evaluation of a scientifically based curriculum (Clements, 2002; Clements & Battista, 2000). Building Blocks is a NSF-funded PreK to grade 2 mathematics curriculum development project, designed to comprehensively address recent standards for early mathematics education for all children (e.g., Clements, Sarama, & DiBiase, 2004; NCTM, 2000). Previous articles describe the design principles behind a set of research-based software microworlds included in the Building Blocks program and the research-based design model that guided its development (Clements, 2002, 2003). This article presents initial summary research on the first set of resulting Preschool mathematics curriculum 3 materials, a research-based, technology-enhanced preschool mathematics curriculum. There have been only a few rigorous tests of the effects of preschool curricula. Some evidence indicates that curriculum can strengthen the development of young students' knowledge of number or geometry (Clements, 1984; Sharon Griffin & Case, 1997; Razel & Eylon, 1991), but no studies of which we are aware have studied the effects of a complete preschool mathematics curriculum, especially on low-income children who are at serious risk for later failure in mathematics (Bowman, Donovan, & Burns, 2001; Campbell & Silver, 1999; Denton & West, 2002; Mullis et al., 2000; Natriello, McDill, & Pallas, 1990; Secada, 1992; Starkey & Klein, 1992). These children possess less mathematical knowledge even before first grade (Denton & West, 2002; Ginsburg & Russell, 1981; Sharon Griffin, Case, & Capodilupo, 1995; Jordan, Huttenlocher, & Levine, 1992; Klein & Starkey, 2004). They receive less support for mathematics learning in the home and school environments, including preschool (Blevins-Knabe & Musun-Miller, 1996; Bryant, Burchinal, Lau, & Sparling, 1994; Farran, Silveri, & Culp, 1991; Holloway, Rambaud, Fuller, & Eggers-Pierola, 1995; Saxe, Guberman, & Gearhart, 1987; Starkey et al., 1999). Rationale for the Building Blocks Project Many curriculum and software publishers claim a research basis for their materials, but the bases of these claims are often dubitable (Clements, 2002). The Building Blocks project is based on the assumption that research-based curriculum development efforts can contribute to (a) more effective curriculum materials because the research reveals critical issues for instruction and contributes information on characteristics of effective curricula to the knowledge base, (b) better understanding of students' mathematical thinking, and (c) research-based change in mathematics curriculum (Clements, Battista, Sarama, & Swaminathan, 1997; Schoenfeld, 1999). Indeed, along with our colleagues, we believe that education will not improve substantially Preschool mathematics curriculum 4 without a system-wide commitment to research-based curriculum and software development (Battista & Clements, 2000; Clements, 2003; Clements & Battista, 2000). Our theoretical framework of research-based curriculum development and evaluation includes three categories and ten methods (Clements, 2002; 2003, see Table 1). Category I, A Priori Foundations, includes three variants of the research-to-practice model, in which extant research is reviewed and implications for the nascent curriculum development effort drawn. (1.) In General A Priori Foundation, broad philosophies, theories, and empirical results on learning and teaching are considered when creating curriculum. (2.) In Subject Matter A Priori Foundation, research is used to identify mathematics that makes a substantive contribution to students' mathematical development, is generative in students’ development of future mathematical understanding, and is interesting to students. (3.) In Pedagogical A Priori Foundation, empirical findings on making activities educationally effective—motivating and efficacious—serve as general guidelines for the generation of activities. In Category II, Learning Model, activities are structured in accordance with empiricallybased models of children’s thinking in the targeted subject-matter domain. This method, (4) Structure According to Specific Learning Model, involves creation of research-based learning trajectories, which we define as “descriptions of children’s thinking and learning...and a related, conjectured route through a set of instructional tasks” (Clements & Sarama, 2004c, p. 83). In Category III, Evaluation, empirical evidence is collected to evaluate the curriculum, realized in some form. The goal is to evaluate the appeal, usability, and effectiveness of an instantiation of the curriculum. (5.) Market Research is commercially-oriented, gathering information about the customer’s needs and preferences. (6.) In Formative Research: Small Group, pilot testing with individuals or small groups of students is conducted on components Preschool mathematics curriculum 5 (e.g., a particular activity, game, or software environment) or on small or large sections of the curriculum. Although teachers are ideally involved in all phases of research and development, the process of curricular enactment is emphasized in the next two methods. Research with a teacher who participated in the development of the materials in method (7) Formative Research: Single Classroom, and then teachers newly introduced to the materials in method (8) Formative Research: Multiple Classrooms, provide information about the usability of the curriculum and requirements for professional development and support materials. Finally, the last two methods, (9) Summative Research: Small Scale and (10) Summative Research: Large Scale, evaluate what can actually be achieved with typical teachers under realistic circumstances in larger contexts with teachers of diverse backgrounds. They use experiments, which provide the most efficient and least biased designs to assess causal relationships (Cook, 2002). Method 9 is employed before 10, due to the need to measure effectiveness in a controlled setting and to the large expense and effort involved in method 10; only effective curricula should be scaled up. Therefore, we employed method 9, Summative Research: Small Scale, in the present study. Given this comprehensive framework of methods, claims that a curriculum is based on research should be questioned to reveal the exact nature between the curriculum and the research used or generated. Unfortunately, there is little documentation of the methods used for most curricula. Often, there is only a hint of A Priori Foundations methods, sometimes non-scientific market research, and minimal formative research with small groups. For example, “beta testing” of educational software is often merely polling of easily accessible peers, conducted late in the process, so that that changes are minimal, given the time and resources dedicated to the project already and the limited budget and pressing deadlines that remain (Char, 1989; Clements & Preschool mathematics curriculum 6 Battista, 2000). In contrast, we designed the Building Blocks approach to incorporate as many of the methods as possible. The next section describes this design. Design of the Building Blocks Materials Previous publications provide detailed descriptions of how we applied these research methods in our design process model (Clements, 2002; Sarama, 2004; Sarama & Clements, 2002); here we provide an overview using the framework previously described. A Priori Foundation methods were used to determine the curriculum’s goals and pedagogy. Based on theory and research on early childhood learning and teaching (Bowman et al., 2001; Clements, 2001), we determined that Building Blocks’ basic approach would be finding the mathematics in, and developing mathematics from, children's activity. The materials are designed to help children extend and mathematize their everyday activities, from building blocks (the first meaning of the project’s name) to art and stories to puzzles. Activities are designed based on children's experiences and interests, with an emphasis on supporting the development of mathematical activity. To do so, the materials integrate three types of media: computers, manipulatives (and everyday objects), and print. Pedagogical foundations were similarly established; for example, we reviewed research on making computer software for young children motivating and educationally effective (Clements, Nastasi, & Swaminathan, 1993; Clements & Swaminathan, 1995; Steffe & Wiegel, 1994). The method of Subject Matter A Priori Foundation was used to determine subject matter content by considering what mathematics is culturally valued (e.g., NCTM, 2000) and empirical research on what constituted the core ideas and skill areas of mathematics for young children (Baroody, 2004; Clements & Battista, 1992; Fuson, 1997), with an emphasis on topics that were mathematical foundational, generative for, and interesting to young children (Clements, Sarama Preschool mathematics curriculum 7 et al., 2004). One of the reasons underlying the name we gave to our project was our desire that the materials emphasize the development of basic mathematical building blocks (the second meaning of the project’s name)—ways of knowing the world mathematically— organized into two areas: (a) spatial and geometric competencies and concepts and (b) numeric and quantitative concepts, based on the considerable research in that domain. Research shows that young children are endowed with intuitive and informal capabilities in both these areas (Baroody, 2004; Bransford, Brown, & Cocking, 1999; Clements, 1999a; Clements, Sarama et al., 2004). For example, research shows that preschoolers know a considerable amount about shapes (Clements, Swaminathan, Hannibal, & Sarama, 1999; Lehrer, Jenkins, & Osana, 1998), and they can do more than we assume, especially working with computers (Sarama, Clements, & Vukelic, 1996). In the broad area of geometry and space, they can do the following: recognize, name, build, draw, describe, compare, and sort twoand three-dimensional shapes, investigate putting shapes together and taking them apart, recognize and use slides and turns, describe spatial locations such as “above” and “behind,” and describe, and use ideas of direction and distance in getting around in their environment (Clements, 1999a). In the area of number, preschoolers can learn to count with understanding (Baroody, 2004; Baroody & Wilkins, 1999; Fuson, 1988; Gelman, 1994), recognize “how many” in small sets of objects (Clements, 1999b; Reich, Subrahmanyam, & Gelman, 1999), and compare numbers (Sharon Griffin et al., 1995). They can count higher and generally participate in a much more exciting and varied mathematics than usually considered (Ginsburg, Inoue, & Seo, 1999; Trafton & Hartman, 1997). Challenging number activities do not just develop children’s number sense; they can also develop children’s competencies in such logical competencies as sorting and ordering (Clements, 1984). Three mathematical themes are woven through both these main areas: (a) patterns, (b) data, and (c) sorting and sequencing. Preschool mathematics curriculum 8 Perhaps the most critical method for Building Blocks was Structure According to Specific Learning Model. All components of the Building Blocks project are based on learning trajectories for each core topic. First, empirically-based models of children’s thinking and learning are synthesized to create a developmental progression of levels of thinking in the goal domain (Clements & Sarama, 2004b; Clements, Sarama et al., 2004; Cobb & McClain, 2002; Gravemeijer, 1999; Simon, 1995). Second, sets of activities are designed to engender those mental processes or actions hypothesized to move children through a developmental progression. We present two examples, one in each of the main domains of number and geometry. The example for number involves addition. Many preschool curricula and practitioners consider addition as inappropriate before elementary school (Clements & Sarama, in press; Heuvel-Panhuizen, 1990; Sarama, 2002; Sarama & DiBiase, 2004). However, research shows that children as young as toddlers can learn simple ideas of addition and subtraction (Aubrey, 1997; Carpenter & Moser, 1984; Clements, 1984; Fuson, 1992a; Groen & Resnick, 1977; Siegler, 1996; Steffe & Cobb, 1988). As long as the situation makes sense to them (Hughes, 1986), preschool children can directly model different types of problems using concrete objects, fingers, and other strategies (Carpenter, Ansell, Franke, Fennema, & Weisbeck, 1993). These child-invented methods, usually using concrete objects and based on subitizing and counting, play a critical developmental role, as the sophisticated counting and composition strategies that develop later are all abbreviations or curtailments of these early solution strategies (Carpenter & Moser, 1984; Fuson, 1992a). Most important for our purpose, reviews of research provide a consistent developmental sequence of the types of problems and solutions in which children can construct solutions (Carpenter & Moser, 1984; for the syntheses most directly related to our work, see Clements & Preschool mathematics curriculum 9 Conference Working Group, 2004; Clements & Sarama, in press; Fuson, 1992a). Selected levels of the resulting addition learning trajectory are presented in Figure 1. The left column, Level, briefly describes each level and the research supporting it. The middle column, Example, provides a behavioral example illustrating that level of thinking. Thus, Non-Verbal Addition is defined as reproducing small sums shown the joining of two groups. For example, many 3-yearolds, after watching 2, then 1 more, button placed under a cloth, can make a collection of 3 to show how many are under the cloth. The following rows describe subsequent developmental levels. The learning trajectory continues past Figure 1 through levels of Counting Strategies, Derived Combinations, and beyond. The next step of building the learning trajectory is to design materials and activities that embody actions-on-objects in a way that mirrors what research has identified as critical mental concepts and processes—children’s cognitive building blocks (the third meaning of the name). These cognitive building blocks are instantiated in onand off-computer activity as actions (processes) on objects (concepts); for example, processes of creating, copying, and combining discrete objects, numbers, or shapes as representations of mathematical ideas. Offering students such objects and actions to be performed on these objects is consistent with the Vygotskian theory that mediation by tools and signs is critical in the development of human cognition (Steffe & Tzur, 1994). Further, designs based on objects and actions force the developer to focus on explicit actions or processes and what they will mean to the students. For the addition trajectory, we designed three offand on-computer activity sets, each with multiple levels: Double Trouble, Dinosaur Shop, and Number Pictures. These sets have the advantage of authenticity as well as serving as a way for children to mathematize these activities (e.g., in setting tables, using different mathematical actions such as establishing one-to-one Preschool mathematics curriculum 10 correspondence, counting and using numerals to represent and generate quantities in the solution of variations of the task). At the Non-Verbal Addition level, the Double Trouble character might place 3 chocolate chips, then 1 more, on a cookie under a napkin. Children put the same number of chips on the other cookie (see the third column in Fig. 1). The teacher conducts similar activity with children using colored paper cookies and brown buttons for “chips.” Similarly, the Dinosaur Shop scenario is used in several contexts. The teacher introduces a dinosaur shop in the socio-dramatic play area and encourages children to count and add during their play. The Small Number Addition row in Figure 1 illustrates a task in which children must move dinosaurs in two boxes into a third and label the sum. Thus, the objects in these and other tasks for the levels described in Figure 1 are single items, groups of items, and numerals. The actions include creating, duplicating, moving, combining, separately, counting, and labeling these objects and groups to solve tasks corresponding to the levels. The example for geometry involves shape composition. We determined that a basic, often neglected, domain of children’s learning of geometry was the composition and decomposition of two-dimensional geometric figures (other domains in geometry include shapes and their properties, transformations/congruence, and measurement). The geometric composition domain was determined to be significant for students in two ways. First, it is a basic geometric competence from building with geometric shapes in the preschool years to sophisticated interpretation and analysis of geometric situations in high school mathematics and above. Second, the concepts and actions of creating and then iterating units and higher-order units in the context of constructing patterns, measuring, and computing are established bases for mathematical understanding and analysis (Clements et al., 1997; Reynolds & Wheatley, 1996; Preschool mathematics curriculum 11 Steffe & Cobb, 1988). The domain is significant to research and theory in that there is a paucity of research on the trajectories students might follow in learning this content. The basic structure of our model of students’ knowledge of shape composition was determined by observations made in the context of early research (Sarama et al., 1996). This was refined through a research review (Mansfield & Scott, 1990; Sales, 1994) and a series of clinical interviews and focused observations by research staff and teachers (Clements, 2001 #1686, leading to the learning trajectory summarized in Figure 2, adapted from Clements, Wilson, & Sarama, 2004). From a lack of competence in composing geometric shapes (Pre-Composer), children gain abilities to combine shapes—initially through trial and error (e.g., Picture Maker) and gradually by attributes—into pictures, and finally synthesize combinations of shapes into new shapes (composite shapes). For example, consider the Picture Maker level in Figure 2. Unlike earlier levels, children concatenate shapes to form a component of a picture. In the top picture in that row, a child made arms and legs from several contiguous rhombi. However, children do not conceptualize the new shapes created (parallelograms) qua geometric shapes. The puzzle task pictured at the bottom of the middle column for that row illustrates a child incorrectly choosing a square because the child is using only one component of the shape, in this case, side length. The child eventually finds this does not work, and completes the puzzle, but only by trial and error. One main instructional task requires children to solve outline puzzles with shapes off and on the computer. Research shows this type of activity to be motivating for young children (Sales, 1994; Sarama et al., 1996). On the computer they play “Shape Puzzles,” illustrated in the third column in Figure 2. The objects are shapes and composite shapes and the actions include creating, duplicating, positioning (with geometric motions), combining, and decomposing both Preschool mathematics curriculum 12 individual shapes (units) and composite shapes (units of units). The characteristics of the tasks require actions on these objects corresponding to each level in the learning trajectory. Note that tasks in these tables are intended to support the developing of the subsequent level of thinking. That is, the instructional task in the Pre-Composer row is assigned to a child operating at the Pre-Composer level and is intended to facilitate the child’s development of competencies at the Piece Assembler level. Ample opportunity for student-led, student designed, open-ended projects are included in each set of activities. Problem posing on the part of students appears to be an effective way for students to express their creativity and integrate their learning (Brown & Walter, 1990; Kilpatrick, 1987; van Oers, 1994), although few empirical studies have been conducted, especially on young children. The computer can offer support for such projects (Clements, 2000). For Shape Puzzles, students design their own puzzles with the shapes; when they click on a “Play” button, their design is transformed into a shape puzzle that either they or their friends can solve. In the addition scenarios, children can make up their own problems with cookies and chips, or dinosaurs and boxes. As another example of a different activity, children design their own “Number Pictures” with shapes and see the resulting combination (Fig. 3; as always, it is also conducted off computer). This activity also illustrates the integration of counting, addition, geometry, and processes such as representation. Our application of formative evaluation methods 5-8 is described in previous publications (Sarama, 2004; Sarama & Clements, 2002). In brief, we tested components of the curriculum and software using clinical interviews and observations of a small number of students to ascertain how children interpreted and understood the objects, actions, and screen design. Next, we tested whether children’s actions-on-objects substantiated the actions of the researchers’ model of Preschool mathematics curriculum 13 children’s mathematical activity and we determined effective prompts to incorporate into each level of each activity. We found that, following any incorrect answer, effective prompts first ask children to try again, and then provide one or more increasingly specific hints, and eventually demonstrate an effective strategy and the correct answer. Asking children to go slowly and try again was successful in a large number of cases; throughout, we strove to give “just enough help” and encourage the child to succeed as independently as possible. However, the specific hints had to be fine-tuned for each activity. (If explicit hints, such as strategy demonstrations, were provided, no assumptions were made about the child’s learning or competence; they were given new problems at the same, or eventually, earlier, levels of the learning trajectory.) Given the activities and prompts, students employed the thinking strategies we had desired. Although teachers were involved in all phases of the design, in methods 7-8 we focused on the process of curricular enactment (Ball & Cohen, 1996), using classroom-based teaching experiments and observing the entire class for information concerning the usability and effectiveness of the software and curriculum. Finally, a content analyses and critical review of the materials at each stage of development was conducted by the advisory board for the project. The following experts studied the materials and provided critiques in meetings twice per year: Arthur J. Baroody, University Of Illinois at Urbana-Champaign; Carol Copple, National Association for the Education of Young Children; Richard Lehrer, Vanderbilt University,; Mary Lindquist, Columbus College; Les Steffe, University of Georgia, and Chuck Thompson, University of Louisville. In summary, we designed the Building Blocks materials upon research in what we consider a well-defined, rigorous, and complete fashion. The main purpose of this study was to evaluate whether materials created according to that model are effective in developing the Preschool mathematics curriculum 14 mathematical knowledge of disadvantaged 4-year-old children and the size of that effect. A secondary purpose was to describe the degree to which the materials developed specific mathematics concepts and skills. To accomplish these two purposes, we used method 9, Summative Research: Small Scale.